Micro-object pose estimation with sim-to-real transfer learning using small dataset

International audience<span style="color: rgb(34, 34, 34); font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Oxygen-Sans, Ubuntu, Cantarell, &quot;Helvetica Neue&quot;, sans-serif; font-size: 18px;"&gtThree-dimensional (3D) pose estimation of micro/nano-objects isessential for the implementation of automatic manipulation inmicro/nano-robotic systems. However, out-of-plane pose estimationof a micro/nano-object is challenging, since the images aretypically obtained in 2D using a scanning electron microscope (SEM)or an optical microscope (OM). Traditional deep learning basedmethods require the collection of a large amount of labeled datafor model training to estimate the 3D pose of an object from amonocular image. Here we present a sim-to-real learning-to-matchapproach for 3D pose estimation of micro/nano-objects. Instead ofcollecting large training datasets, simulated data is generated toenlarge the limited experimental data obtained in practice, whilethe domain gap between the generated and experimental data isminimized via image translation based on a generative adversarialnetwork (GAN) model. A learning-to-match approach is used to mapthe generated data and the experimental data to a low-dimensionalspace with the same data distribution for different pose labels,which ensures effective feature embedding. Combining the labeleddata obtained from experiments and simulations, a new trainingdataset is constructed for robust pose estimation. The proposedmethod is validated with images from both SEM and OM, facilitatingthe development of closed-loop control of micro/nano-objects withcomplex shapes in micro/nano-robotic systems.</span&g

Generalist Vision Foundation Models for Medical Imaging: A Case Study of Segment Anything Model on Zero-Shot Medical Segmentation

In this paper, we examine the recent Segment Anything Model (SAM) on medical images, and report both quantitative and qualitative zero-shot segmentation results on nine medical image segmentation benchmarks, covering various imaging modalities, such as optical coherence tomography (OCT), magnetic resonance imaging (MRI), and computed tomography (CT), as well as different applications including dermatology, ophthalmology, and radiology. Those benchmarks are representative and commonly used in model development. Our experimental results indicate that while SAM presents remarkable segmentation performance on images from the general domain, its zero-shot segmentation ability remains restricted for out-of-distribution images, e.g., medical images. In addition, SAM exhibits inconsistent zero-shot segmentation performance across different unseen medical domains. For certain structured targets, e.g., blood vessels, the zero-shot segmentation of SAM completely failed. In contrast, a simple fine-tuning of it with a small amount of data could lead to remarkable improvement of the segmentation quality, showing the great potential and feasibility of using fine-tuned SAM to achieve accurate medical image segmentation for a precision diagnostics. Our study indicates the versatility of generalist vision foundation models on medical imaging, and their great potential to achieve desired performance through fine-turning and eventually address the challenges associated with accessing large and diverse medical datasets in support of clinical diagnostics.Comment: Published in Diagnostic

Large AI Models in Health Informatics: Applications, Challenges, and the Future

Large AI models, or foundation models, are models recently emerging with massive scales both parameter-wise and data-wise, the magnitudes of which can reach beyond billions. Once pretrained, large AI models demonstrate impressive performance in various downstream tasks. A prime example is ChatGPT, whose capability has compelled people's imagination about the far-reaching influence that large AI models can have and their potential to transform different domains of our lives. In health informatics, the advent of large AI models has brought new paradigms for the design of methodologies. The scale of multi-modal data in the biomedical and health domain has been ever-expanding especially since the community embraced the era of deep learning, which provides the ground to develop, validate, and advance large AI models for breakthroughs in health-related areas. This article presents a comprehensive review of large AI models, from background to their applications. We identify seven key sectors in which large AI models are applicable and might have substantial influence, including 1) bioinformatics; 2) medical diagnosis; 3) medical imaging; 4) medical informatics; 5) medical education; 6) public health; and 7) medical robotics. We examine their challenges, followed by a critical discussion about potential future directions and pitfalls of large AI models in transforming the field of health informatics.Comment: This article has been accepted for publication in IEEE Journal of Biomedical and Health Informatic

Egocentric Image Captioning for Privacy-Preserved Passive Dietary Intake Monitoring

Camera-based passive dietary intake monitoring is able to continuously capture the eating episodes of a subject, recording rich visual information, such as the type and volume of food being consumed, as well as the eating behaviours of the subject. However, there currently is no method that is able to incorporate these visual clues and provide a comprehensive context of dietary intake from passive recording (e.g., is the subject sharing food with others, what food the subject is eating, and how much food is left in the bowl). On the other hand, privacy is a major concern while egocentric wearable cameras are used for capturing. In this paper, we propose a privacy-preserved secure solution (i.e., egocentric image captioning) for dietary assessment with passive monitoring, which unifies food recognition, volume estimation, and scene understanding. By converting images into rich text descriptions, nutritionists can assess individual dietary intake based on the captions instead of the original images, reducing the risk of privacy leakage from images. To this end, an egocentric dietary image captioning dataset has been built, which consists of in-the-wild images captured by head-worn and chest-worn cameras in field studies in Ghana. A novel transformer-based architecture is designed to caption egocentric dietary images. Comprehensive experiments have been conducted to evaluate the effectiveness and to justify the design of the proposed architecture for egocentric dietary image captioning. To the best of our knowledge, this is the first work that applies image captioning to dietary intake assessment in real life settings

Nonadiabatic effects in a generalized Jahn-Teller lattice model: heavy and light polarons, pairing and metal-insulator transition

The ground state polaron potential of 1D lattice of two-level molecules with spinless electrons and two Einstein phonon modes with quantum phonon-assisted transitions between the levels is found anharmonic in phonon displacements. The potential shows a crossover from two nonequivalent broad minima to a single narrow minimum corresponding to the level positions in the ground state. Generalized variational approach implies prominent nonadiabatic effects:(i) In the limit of the symmetric E-e Jahn- Teller situation they cause transition between the regime of the predominantly one-level "heavy" polaron and a "light" polaron oscillating between the levels due to phonon assistance with almost vanishing polaron displacement. It implies enhancement of the electron transfer due to decrease of the "heavy" polaron mass (undressing) at the point of the transition. Pairing of "light" polarons due to exchange of virtual phonons occurs. Continuous transition to new energy ground state close to the transition from "heavy" polaron phase to "light" (bi)polaron phase occurs. In the "heavy" phase, there occurs anomalous (anharmonic) enhancements of quantum fluctuations of the phonon coordinate, momentum and their product as functions of the effective coupling. (ii) Dependence of the polaron mass on the optical phonon frequency appears.(iii) Rabi oscillations significantly enhance quantum shift of the insulator-metal transition line to higher values of the critical effective e-ph coupling supporting so the metallic phase. In the E-e JT case, insulator-metal transition coincide with the transition between the "heavy" and the "light" (bi)polaron phase at certain (strong) effective e-ph interaction.Comment: Paper in LaTex format (file jtseptx.tex) and 9 GIF-figures (ppic_1.gif,...ppic_9.gif

Alternative-Splicing in the Exon-10 Region of GABAA Receptor β2 Subunit Gene: Relationships between Novel Isoforms and Psychotic Disorders

BACKGROUND: Non-coding single nucleotide polymorphisms (SNPs) in GABRB2, the gene for beta(2)-subunit of gamma-aminobutyric acid type A (GABA(A)) receptor, have been associated with schizophrenia (SCZ) and quantitatively correlated to mRNA expression and alternative splicing. METHODS AND FINDINGS: Expression of the Exon 10 region of GABRB2 from minigene constructs revealed this region to be an "alternative splicing hotspot" that readily gave rise to differently spliced isoforms depending on intron sequences. This led to a search in human brain cDNA libraries, and the discovery of two novel isoforms, beta(2S1) and beta(2S2), bearing variations in the neighborhood of Exon-10. Quantitative real-time PCR analysis of postmortem brain samples showed increased beta(2S1) expression and decreased beta(2S2) expression in both SCZ and bipolar disorder (BPD) compared to controls. Disease-control differences were significantly correlated with SNP rs187269 in BPD males for both beta(2S1) and beta(2S2) expressions, and significantly correlated with SNPs rs2546620 and rs187269 in SCZ males for beta(2S2) expression. Moreover, site-directed mutagenesis indicated that Thr(365), a potential phosphorylation site in Exon-10, played a key role in determining the time profile of the ATP-dependent electrophysiological current run-down. CONCLUSION: This study therefore provided experimental evidence for the importance of non-coding sequences in the Exon-10 region in GABRB2 with respect to beta(2)-subunit splicing diversity and the etiologies of SCZ and BPD

Controlling crystallization and its absence: Proteins, colloids and patchy models

The ability to control the crystallization behaviour (including its absence) of particles, be they biomolecules such as globular proteins, inorganic colloids, nanoparticles, or metal atoms in an alloy, is of both fundamental and technological importance. Much can be learnt from the exquisite control that biological systems exert over the behaviour of proteins, where protein crystallization and aggregation are generally suppressed, but where in particular instances complex crystalline assemblies can be formed that have a functional purpose. We also explore the insights that can be obtained from computational modelling, focussing on the subtle interplay between the interparticle interactions, the preferred local order and the resulting crystallization kinetics. In particular, we highlight the role played by ``frustration'', where there is an incompatibility between the preferred local order and the global crystalline order, using examples from atomic glass formers and model anisotropic particles.Comment: 11 pages, 7 figure

The Origin of Behavior

We propose a single evolutionary explanation for the origin of several behaviors that have been observed in organisms ranging from ants to human subjects, including risk-sensitive foraging, risk aversion, loss aversion, probability matching, randomization, and diversification. Given an initial population of individuals, each assigned a purely arbitrary behavior with respect to a binary choice problem, and assuming that offspring behave identically to their parents, only those behaviors linked to reproductive success will survive, and less reproductively successful behaviors will disappear at exponential rates. When the uncertainty in reproductive success is systematic, natural selection yields behaviors that may be individually sub-optimal but are optimal from the population perspective; when reproductive uncertainty is idiosyncratic, the individual and population perspectives coincide. This framework generates a surprisingly rich set of behaviors, and the simplicity and generality of our model suggest that these derived behaviors are primitive and nearly universal within and across species

Social Cognitive Role of Schizophrenia Candidate Gene GABRB2

10.1371/journal.pone.0062322PLoS ONE84

A Study of Time-Dependent CP-Violating Asymmetries and Flavor Oscillations in Neutral B Decays at the Upsilon(4S)

We present a measurement of time-dependent CP-violating asymmetries in neutral B meson decays collected with the BABAR detector at the PEP-II asymmetric-energy B Factory at the Stanford Linear Accelerator Center. The data sample consists of 29.7 ${\rm fb}^{-1}$ recorded at the $\Upsilon(4S)$ resonance and 3.9 ${\rm fb}^{-1}$ off-resonance. One of the neutral B mesons, which are produced in pairs at the $\Upsilon(4S)$, is fully reconstructed in the CP decay modes $J/\psi K^0_S$, $\psi(2S) K^0_S$, $\chi_{c1} K^0_S$, $J/\psi K^{*0}$ ($K^{*0}\to K^0_S\pi^0$) and $J/\psi K^0_L$, or in flavor-eigenstate modes involving $D^{(*)}\pi/\rho/a_1$ and $J/\psi K^{*0}$ ($K^{*0}\to K^+\pi^-$). The flavor of the other neutral B meson is tagged at the time of its decay, mainly with the charge of identified leptons and kaons. The proper time elapsed between the decays is determined by measuring the distance between the decay vertices. A maximum-likelihood fit to this flavor eigenstate sample finds $\Delta m_d = 0.516\pm 0.016 {\rm (stat)} \pm 0.010 {\rm (syst)} {\rm ps}^{-1}$. The value of the asymmetry amplitude $\sin2\beta$ is determined from a simultaneous maximum-likelihood fit to the time-difference distribution of the flavor-eigenstate sample and about 642 tagged $B^0$ decays in the CP-eigenstate modes. We find $\sin2\beta=0.59\pm 0.14 {\rm (stat)} \pm 0.05 {\rm (syst)}$, demonstrating that CP violation exists in the neutral B meson system. (abridged)Comment: 58 pages, 35 figures, submitted to Physical Review